SUPPORT / SAMPLES & SAS NOTES
 

Support

Problem Note 45818: Slow performance when computing robust estimators of scale

DetailsHotfixAboutRate It

A noticeable performance degradation might be observed on 64-bit operating systems other than Windows as compared to Windows operating systems when robust estimators of scale are requested in PROC UNIVARIATE. The robust estimators of scale are Gini's Mean Difference, the Interquartile Range, MAD, Qn, and Sn. When any of one of these estimators or its standard deviation is requested, they are all computed by the procedure internally. These statistics are requested in either of the following ways:


  • Specifying the ROBUSTSCALE option in the PROC UNIVARIATE statement
  • Specifying any of the following keywords on an INSET or OUTPUT statement:
    • GINI
    • MAD
    • QN
    • SN
    • STD_GINI
    • STD_MAD
    • STD_QN
    • STD_QRANGE
    • STD_SN

It is the Qn computation that takes the longest and causes very slow performance. When the robust estimators are specified, there is an internal check that omits the computation of Qn and sets it to a missing value when the number of observations exceeds a preset value. On the Windows operating system, Qn is not computed when there are more than approximately 65,530 observations. However, on 64-bit operating systems other than Windows, it is not until the number of observations exceeds approximately 134,200,000 that the Qn computation is suppressed. Therefore, the Qn computation is carried out for much larger data sets on these 64-bit machines.

A Hot Fix is available that eliminates the computation of QN and STD_QN in cases where they are not specifically requested. The fix improves performance when any of GINI, MAD, SN, STD_GINI, STD_MAD, STD_QRANGE and STD_SN are requested, and QN and STD_QN are not requested. With the Hot Fix installed, only the requested robust estimators of scale are computed.

The Hot Fix causes no change in the performance of PROC UNIVARIATE if no robust estimators of scale are requested. There is also no change in the performance when QN or STD_QN are requested.



Operating System and Release Information

Product FamilyProductSystemProduct ReleaseSAS Release
ReportedFixed*ReportedFixed*
SAS SystemBase SASMicrosoft Windows Server 2003 Enterprise 64-bit Edition9.21_M39.3_M29.2 TS2M39.3 TS1M2
Microsoft Windows XP 64-bit Edition9.21_M39.3_M29.2 TS2M39.3 TS1M2
Microsoft Windows XP Professional9.21_M39.3_M29.2 TS2M39.3 TS1M2
Microsoft Windows Server 2008 for x649.21_M39.3_M29.2 TS2M39.3 TS1M2
Microsoft® Windows® for 64-Bit Itanium-based Systems9.21_M39.3_M29.2 TS2M39.3 TS1M2
Microsoft Windows Server 2003 for x649.21_M39.3_M29.2 TS2M39.3 TS1M2
Microsoft Windows Server 2003 Standard Edition9.21_M39.3_M29.2 TS2M39.3 TS1M2
Microsoft Windows Server 2003 Enterprise Edition9.21_M39.3_M29.2 TS2M39.3 TS1M2
Microsoft Windows Server 2003 Datacenter Edition9.21_M39.3_M29.2 TS2M39.3 TS1M2
Microsoft® Windows® for x649.21_M39.3_M29.2 TS2M39.3 TS1M2
Microsoft Windows Server 2003 Datacenter 64-bit Edition9.21_M39.3_M29.2 TS2M39.3 TS1M2
Solaris for x649.21_M39.3_M29.2 TS2M39.3 TS1M2
Linux for x649.21_M39.3_M29.2 TS2M39.3 TS1M2
64-bit Enabled Solaris9.21_M39.3_M29.2 TS2M39.3 TS1M2
Windows Vista for x649.21_M39.3_M29.2 TS2M39.3 TS1M2
Windows 7 Ultimate x649.21_M39.3_M29.2 TS2M39.3 TS1M2
Windows 7 Professional x649.21_M39.3_M29.2 TS2M39.3 TS1M2
Windows 7 Home Premium x649.21_M39.3_M29.2 TS2M39.3 TS1M2
64-bit Enabled AIX9.21_M39.3_M29.2 TS2M39.3 TS1M2
Windows 7 Enterprise x649.21_M39.3_M29.2 TS2M39.3 TS1M2
64-bit Enabled HP-UX9.21_M39.3_M29.2 TS2M39.3 TS1M2
* For software releases that are not yet generally available, the Fixed Release is the software release in which the problem is planned to be fixed.